A Recurrent Self-Organizing Map for Temporal Sequence Processing
نویسندگان
چکیده
We present a novel approach to unsupervised temporal sequence processing in the form of an unsupervised, recurrent neural network based on a selforganizing map (SOM). A standard SOM clusters each input vector irrespective of context, whereas the recurrent SOM presented here clusters each input based on an input vector and a context vector. The latter acts as a recurrent conduit feeding back a 2-D representation of the previous winning neuron. This recurrency allows the network to operate on temporal sequence processing tasks. The network has been applied to the difficult natural language processing problem of position variant recognition, e.g. recognising a noun phrase regardless of its position within a sentence.
منابع مشابه
Temporal sequence processing using recurrent SOM
Recurrent Self-Organizing Map (RSOM) is studied in temporal sequence processing. RSOM includes a recurrent difference vector in each unit of the map, which allows storing temporal context from consecutive input vectors fed to the map. RSOM is a modification of the Temporal Kohonen Map (TKM). It is shown that RSOM learns a correct mapping from temporal sequences of a simple synthetic data, while...
متن کاملContext Learning with the Self Organizing Map
In this paper a Recurrent Self-Organizing Map (RSOM) algorithm is proposed for temporal sequence processing. The RSOM algorithm is close in nature to the Kohonen's Self-Organizing Map, except that in the RSOM context of the temporal sequence is involved both in the best matching unit nding and in the adaptation of the weight vectors of the map via an introduced recursive di erence equation asso...
متن کاملNeural Gas for Sequences
For unsupervised sequence processing, standard self organizing maps (SOM) can be naturally extended by recurrent connections and explicit context representations. Known models are the temporal Kohonen map (TKM), recursive SOM, SOM for structured data (SOMSD), and HSOM for sequences (HSOM-S). We discuss and compare the capabilities of exemplary approaches to store different types of sequences. A...
متن کاملSelf-organizing maps in sequence processing
Models are abstractions of observed real world phenomena or processes. A good model captures the essential properties of the modeled phenomena. In the statistical learning paradigm the processes that generate observations are assumed unknown and too complex for analytical modeling, thus the models are trained from more general templates with measured observations. A substantial part of the proc...
متن کاملSelf-Organizing Maps for Time Series
We review a recent extension of the self-organizing map (SOM) for temporal structures with a simple recurrent dynamics leading to sparse representations, which allows an efficient training and a combination with arbitrary lattice structures. We discuss its practical applicability and its theoretical properties. Afterwards, we put the approach into a general framework of recurrent unsupervised m...
متن کامل